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SYSTEM AND METHOD FOR PATTERN IDENTIFICATION 



BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

This invention relates to machine vision, and more particularly, to a system and method 
for pattern identification. 

2. Description of the Related Art 

Automated manufacturing processes often rely on machine vision systems to identity and 
locate parts that are being assembled. This allov^s a machine, such as a robot, to interact with 
( e.R. , pick up) the individual parts using pattem recognition. For example, the pattern could be 
an image of the part, such as an image of a keypad of a cellular phone unit that is to be assembled 
in a housing. A linear pattem, such as a cross hair, may also be used to aid in precision 
alignment. 

Various methods of machine vision positioning are currently practiced, including 
variations based on the general Hough transform and variations based on correlation (template 
matching). Correlation may use binary exclusive-OR correlation and, more recently, gray-scale 
normalized correlation. One common element of these systems is that they require, as input, an 
image template of the part whose pattern is being located. 

The Hough transform, however, is inefficient when the part to be located is subject to 
rotation because, generally, multiple analyses must be performed using different potential angles 
of rotation. Specifically, for each possible angle of rotation, the Hough transform generally 
requires a separate reference table. Accordingly, if a user wants to have ftiU rotation search 
capability at ten degree increments in angles of rotation, then the Hough transform would require 



a minimum of thirty-six tables, and then sequential searches would have to be performed on each 
table. Because such a analysis is computationally demanding, the search time is significantly 
increased. 

SUMMARY OF THE INVENTION 

Therefore, a need has arisen for a system and method for pattem identification that 
overcomes these and other shortcomings of the related art. 

According to one embodiment of the present invention, a method for locating a pattem is 
disclosed. The method includes the steps of (1) providing a pattem image corresponding to the 
pattem to be located; (2) extracting a pattem contour from the pattem image; (3) generating 
vector information for the pattern contours, relative to a reference point; (4) creating a reference 
table for storing the vector information, the reference tables corresponding to the pattem contour; 
(5) providing a scene image, which will be searched for the pattem; (6) extracting a scene 
contour from the scene image; (7) generating vector information for the scene contours; and (8) 
determining whether the pattem has been located within the scene image using the reference 
table and the vector information for the scene contour, and if so, identifying a location of the 
pattem within the scene image and an angle of rotation of the pattem within the scene image. 

In another embodiment, a method for pattem recognition comprises the following steps: 
(1) extracting pattem vector information from at least one pattem image, each pattem image 
having a pattem reference point; (2) creating a reference table containing the pattem vector 
information for each of the at least one pattem image; (3) extracting scene contour information 
from a scene image; (4) calculating a potential reference point based on the scene contour 
information and the reference table; (5) matching the potential reference point with one of the at 
least one pattem reference points; and (6) identifying a pattem image corresponding to the 
matching pattem reference point. 

In still another embodiment, a system for pattem identification is disclosed. The system 
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includes a first image capture device that captures a pattern image which includes an image of a 
pattern; a second image capture device that captures a scene image to be searched for the pattern; 
a processor processing the images, which includes means for extracting at least one pattem 
contour from the pattem image; means for generating vector information for each of said at least 
one pattem contours, relative to a reference point; means for creating at least one reference table 
for storing vector information, each of said at least one reference tables corresponding to at least 
one pattem contour; a means for extracting at least one scene contour firom a scene image; means 
for generating vector information for each of said at least one scene contours; and means for 
locating the pattem image within the scene image using the at least one reference tables and the 
vector information for the at least one scene contours. 

BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete understanding of the present invention, the objects and advantages 
thereof, reference is now made to the following descriptions taken in connection with the 
accompanying drawings in which: 

Fig. 1 depicts an example of a pattem; 

Fig, 2 illustrates an illustration of a contour according to one embodiment of the present 
invention; 

Fig, 3 illustrates a flowchart of a method for pattem identification according to one 
embodiment of the present invention; 

Fig. 4 illustrates a flowchart of a training process; 
Fig. 5 illustrates a flowchart of a search process; 

Fig. 6a illustrates an example of a pattem (x, y) location image according to one 
embodiment of the present invention; 

Fig. 6b illustrates an example of a pattern angle image according to one embodiment of 
the present invention; 
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Fig. 7 illustrates an example of a pattern image to be trained; 

Fig, 8 illustrates the pattern image of Fig. 7 including index points, front and back points, 
and stick vectors; 

Fig. 9 illustrates a graphical plot of the points identified in Fig. 8; 
5 Fig. 10 illustrates scene image to be searched for the pattern shown in Fig. 7; 

Fig. 11 illustrates a plot of the index points and front and back points of the scene image 
of Fig. 10; and 

Fig. 12 illustrates an image of the accumulator for the scene image of Fig. 10. 

10 O DETAILED DESCRIPTION OF THE INVENTION 

01 The preferred embodiment of the present invention and its advantages are best understood 

S by referring to Figs. 1 through 12 of the drav^ings, like numerals being used for like and 

corresponding parts of the various drawings. 
^ For convenience, the present invention will be described with the following convention, 

15 !^ with reference to image 102 shown in Fig. 1. Image 102 may be a digital image of a part or an 
^ object. In the figure, image 102 is a digital image of a shutter speed selection knob for a camera. 
G It may be desirable for a machine to determine the precise orientation of this part (such as its 
rotation) so that a machine may properly interact with this part during assembly. 

Image 102 may include one or more pattems 103. In the figure, image 102 includes 
20 several pattems 103, including, for example, the outer cylindrical surface of the knob, the hollow 

interior of the knob, as well as the various numbers and letters on the knob indicating different 
shutter speeds. 

Pattern 103 may also include at least one contour 104. In general, contour 104 may be 
thought of as being an ordered list of edge point coordinates that describe a boundary of a 
25 pattem, including both internal and external boundaries. In the figure, for example, contour 104 

is indicated at the outer edge of the knob. 
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Generally, the present invention "trains" on image 102 so that the invention will know the 
pattern of the part for which the invention is looking. During the training process, the present 
invention is given, or creates, a "pattern image," and the system trains on the pattern image. 
During the searching process, the present invention searches a "scene image" in an effort to 
locate the pattern that was used to train the system. So, when the term "pattern image" is used, 
this application is referring to the image used in training, and when the term "scene image" is 
used, this application is referring to the search process. 

A brief discussion of the theory of the present invention is provided. Fig. 2 illustrates a 
contour that has been extracted, for example, from a pattern image during the training process. 
Referring to Fig. 2, contour 202 includes a plurality of contour points 204, which Ue along 
contour 202. Index point 206 (located at (xq, yo)), which is a contour point, is selected, and two 
reference points, back point 208 (located at (xi, yi), and "behind" index point 206 by a constant 
k) and front point 210 (located at (x2, yi), and "in front of index point 206 by a constant k (not 
shown)) are selected. Variable k controls the locality of the curvature measurement. In one 
embodiment, k represents the number of contour points that separate back point 208 from index 
point 206, which is the same as the number of contour points separating front point 210 from 
index point 206. A smaller value for k gives very local curvature measurements, while a larger 
value for k gives more global measurements. 

In one embodiment, indices of front point 208 and back point 210 may be selected 
automatically. In general, they are separated from index of index point 206 by k ( i.e. , each is k 
data points away from index point 206). The index of front point 208 is smaller than the index of 
index point 206 by k, while the index of back point 210 is larger than the index of index point 
206 by k. For example, if (xo, yo) is at index 100, then back point 208 is the point at index 90, 
and front point 210 is at index 1 10 (for ^=10). 

Generally, k is chosen based upon the severity of the curves in the pattem contours and 
based upon the number of contour points that are used to represent the contour. Generally, a 
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smaller k may be preferred, but k must not be too small. A /: of 10 will work in most standard 
applications. 

Stick vector 212, which is a vector that connects back point 208 and front point 210, has a 
direction representing the direction of "crawl;" that is, the direction from back point 208 to front 
5 point 210. Stick vector 212 has an angle of Go relative to the horizontal axis. 

In another embodiment, an additional back point (not shown) and an additional front 
point (not shown) may also be used. In general, these points may be indexed with a value greater 
than k. These points may be used to define an additional stick vector (not shown), which may 
provide additional information regarding contour 202. 
10 ^ Line 209 connects index point 206 and back point 208, while line 211 connects index 

^ point 206 and front point 210. Angle 02 represents the angle between stick 212 and line 211. 

55 

G Algorithm reference point 214, located at (xc, yc), niay be selected by a user, or it may be 

Ln the centroid of the pattem, as will be discussed in greater detail below. 

% A distance h represents a contour curvature measure for contour point 206. The distance 

15 g h is the shortest distance from index point 206 to stick 212. For example, for an appropriate k 
m value, a value ofh=0 indicates that the stick is faUing on a straight line. 

g Vector R may be constructed by drawing a vector from contour point 206 to algorithm 

reference point 214. This vector has magnitude of d, a direction pointing from contour point 206 
to algorithm reference point 214, and angle 0i relative to the horizontal axis. Angle 56 is the 

20 angle between R and stick vector 212. Angle 59 may also be referred to as the "SR Angle." 

Angle 50 is rotation invariant. This means that regardless of any rotation of contour 202 (or 
rotation of the part/object) relative to the horizontal axis, this angle is the same for a particular 
point on contour 202. Because angle 50 is rotation invariant, it is an ideal variable, as it reduces 
search time significantly. It also eUminates the need for a separate table for each angle of 

25 rotation. 

Angle 50Gradient IS the angle between gradient 216 and vector R. This may also be referred 



to as the "GR Angle." In general, SQoradient determines the position of algorithm reference point 
214 relative to stick vector 212 ( i.e. , (on what side of stick vector 212) so long as contour 202 is 
crawled on in the same direction when training and when searching. Crawling in the same 
direction may not happen in every image, due to disconnection or extra connection of a contour. 

In general, the lengths and angles identified above may be determined by trigonometry, or 
by other higher-level mathematics. 

With the convention and theory described above in mind, a flowchart of a method for 
pattern identification according to one embodiment of the present invention is provided in Fig. 3. 
First, in step 302, the system is ''trained" on a pattern image. Referring to Fig. 4, a flowchart of 
the training process is provided. First, in step 402, pattern contours are extracted from the 
pattern image. In one embodiment, this may be achieved by detecting the levels of the edges on 
the pattern image, which computes the optimal value for the edge level ( i.e. , the edge strength) in 
order to filter out noise edge pixels. Other filters may be used as desired. 

Pattern contours may be extracted from the pattern image, preferably after any 
conditioning. In one embodiment, an extraction engine, such as EdgeCrawler, developed by 
Imaging Technology, Inc., of Bedford, Massachusetts, may be used to extract the contours from 
the pattern image. The extraction engine may extract edge pixels by running a Gaussian filter 
followed by a Sobel transform. Non-maxima suppression may be performed on all edge points 
above the value of the level detected by the edge level detection. In another embodiment, a top- 
down crawler may be used to create a contour from the edges. 

In another embodiment, the edges may be extracted by locating at least one edge of the 
pattern image of the scene image and recording a starting point for the edge. The edge is then 
"crawled" along; that is, each pixel along the edge is analyzed. As the pixels are analyzed, they 
may be extracted from the edge, and then may be filtered. A contour may then be created from 
the filtered and extracted pixels. 

Next, in step 404, the extracted contours may be conditioned. This step helps to identify 



and add undetected edge points, if any. In addition, aliased edge points may be removed. In one 
embodiment, each contour may be spatially filtered with a 5 x 1 filter. Of course, different filters 
may be used. Each contour may then be normaUzed to create a new set of contours in which the 
distance between any neighboring points is 1 .0 (one) pixel. Raw contour points may have a 
spacing that varies between 1 and 1.41 pixels in distance, depending on the angle between points. 
The result of the contour conditioning stage is at least one pattern contour. 

In step 406, an algorithm reference (x, y) position may be selected. Step 406 may be 
exercised before step 404, or even after step 408. In one embodiment, the reference point may be 
the centroid coordinate, or central moment, of the pattern contour(s). At the end of searching, the 
algorithm reference point may be translated back to user coordinates. 

In step 408, the pattern contour may be optimized. This may be accomplished by setting 
levels based on the analysis of pattern contours, including the number of contour points, the 
curvature statistics of contour points, etc. These points may be used to determine the values of 
each parameter. 

In step 410, a reference table is created. This may be performed simultaneously with step 
408. This may be accompUshed by analyzing every contour in the pattern contours, and for every 
point on the contour, extracting the contour curvature, h, the vector R (r, 0i), the angle between 
the R vector and the stick vector 5G, and the angle 50Gradient. In one embodiment, these values 
may be grouped in a reference table according to their contour curvature h in order to decrease 
search time, and to better shape the peak shape in the pattem location image, discussed below. 

An example of a reference table is provided below, wherein each rectangle may include 
data represented generically by variable. The fact that the identical variables are shown in the 
table is not intended to suggest that each measurement for the variables are identical. 
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Referring again to Fig. 3, in step 304, once the training is complete, the reference table 
ly and related information may be stored in a database. In one embodiment, the database may 
5 Iry include a plurality of reference tables for different pattern images, 
m In step 306, a scene image is acquired. In one embodiment, the scene image may be 

G acquired in real-time; in other embodiments, the scene image may be acquired from a database, 

O or provided to the system form an outside source. 

D 

In step 308, the scene image is searched. Referring to Fig. 5, a flowchart of the search 
10 process is provided. In step 502, contours are extracted from the scene image. In one 
embodiment, contours may be extracted from the scene image in the same way that contours 
were extracted from the pattern image. Therefore, a detailed description of this step is omitted. 

In step 504, the contours may be conditioned and/or normalized. In one embodiment, 
contours may be conditioned and/or normalized from the scene image in the same way that 
15 contours were conditioned and/or normalized from the pattern image. Therefore, a detailed 

description of this step is omitted. 

In step 506, as the contour points are crawled, the values of h. Go, and Goradiem for each 
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contour point are determined. 

In step 508, the location of a potential reference point is computed from scene 9o (the 
angle between a scene stick and the horizontal axis) and R (r, 59 1) of a cell in the table. In doing 
so, the scene contour curvature value, h, may be used to select a column of the table. A potential 
5 reference point for the scene pattern is calculated for each table cell in that column (all the points 
with the same contour curvature value h). 

The potential reference point (x, y) is entered into a reference point accumulator which 
"counts" each instance of a potential reference point. In one embodiment, a pattern location 
image, in which the pixel value at the reference coordinates is increment-ed each time a 
10 calculation yields a reference point with that coordinate, is used as the accumulator. In other 

!r: words, the "height" of a particular coordinate increases each time the particular coordinate is 
y calculated as a potential reference point. In one embodiment, this step may be repeated for each 
Ul of the scene contour points. 

An example of a pattem location image is provided in Fig. 6a. In Fig. 6a, the tallest 
15 O column reflects an accumulator that shows the greatest number of "hits," and hence, the greater 
iijl likelihood of a match. 

p Referring again to Fig. 5, in step 510, the angle of rotation of the pattem may be 

computed. This angle may be computed based upon the angle difference between 9o (stick vector 
angle) of scene point, and 9o in the table cell. This angle is also entered into an angle 
20 accumulator, which tallies each instance of an angle. In one embodiment, a pattem angle image 

in which the angle value is stored (stacked) at computed (x, y) coordinates may be used. 

In another embodiment, the angle difference between Ocradient of the scene point, and 
Ocradient in the table cell may be used. In general, however, 9o may be more stable than 9Gradient- 
An example of a pattem angle image is provided in Fig. 6b. 
25 Referring again to Fig. 5, in step 512, after all points are processed, the accimiulator for 

the potential reference point is analyzed to determine the best potential reference point. 
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Next, in step 514, the angle of rotation is determined. In one embodiment, a clustering 
algorithm, may be used to determine the angle of rotation. The clustering algorithm may return a 
predetermined number of strong "peaks," as well as smaller peaks near a strong peak. The 
clustering algorithm may average the peaks to return the most likely angle of rotation. 

In step 516, edges may be verified. After the best potential reference point and rotation 
angle for the pattem are identified, the position of all edge points for the potential reference point 
(forming a potential pattem) may be translated into the coordinates of the scene. This may be 
done by moving a position of the potential pattem to the potential reference point, and then 
rotating the potential pattem using the angle retumed by the search. Depending on the accuracy, 
the two sets of edge points (edge points for the potential pattem, and edge points for the pattem) 
may overlap spatially. The movement and the rotation should cause the edges of the potential 
pattem and the pattem image to overlap. The verification procedure verifies that, for each model 
edge point, there is a corresponding edge point in the scene. 

Through processing all the points, two numbers can be measured: the number of edge 
points that verified positive ( i.e. , aligned or matched), and number of points that verified negative 
(i.e. , did not align or did not match). These numbers may be used to compute a "score," which 
represents a similarity measure between the potential pattem edges and the pattem edges at given 
point and angle. For example: if the number of positive edge points (A^) is 1000, and the 
number of negative edge points (Nn) is 100, the "score" may be determined as: 



In the example, the score is .-09 0 9', or 90.9%. The score may be used to determine 



whether or not the potential reference point is satisfactory. 

It should be noted that the present invention may be implemented using an image capture 
device, such as a digital camera, as well as a processor. An example of a suitable processor is a 



Score = 



Np 



Np + Nn 
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Pentiuin®-series processor, manufactured by Intel, Inc., Santa Clara, California. In another 
embodiment, an application-specific integrated circuit (ASIC) may be used. In still another 
embodiment, a digital signal processor (DSP) may be used. Other processor types may be used 
as desired. 

EXAMPLE 

In order to better understand the present invention, an example is provided. This example 
does not limit the present invention in any way, and is intended to illustrate one embodiment of 
the present invention. 

The example may be understood by referring to Figs. 7-12. Fig, 7 depicts a pattem that is 
to be trained; specifically, this figure includes sample shutter part 702. Referring to Fig. 8, 
shutter 702 has reference point 802, as well as index points 804, 806, 808, 810, 812, and 814. 
Each index point i (xO/, yO^) has a front point (x2/, y2/) and a back point (xl/, yl/), which are each 
connected by a stick vector. 

If the coordinates for reference point 802 are xc= 284 and yc= 192, then the coordinates 
of the index points z, with their respective back points and front points, are: 

The points back points front points 



xO.:= 


yO,:= 


xl.:= 
1 




x2.:= 




211.51 




198,79 




209.60 




214.65 




226.55 




193.37 


272.20 




255.50 




287.68 




257.52 




257.44 




251.46 


310.51 




252.06 




315.91 
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295.49 




258.13 


315.12 




196.70 




314.50 




186.36 




318.16 




211.00 


337.74 




143.04 




326.81 




129.85 




338.54 




156.32 


271.44 




164.44 




262.71 




174.34 




280.55 




151.78 



These points are plotted in the graph of Fig. 9, with the index points plotted as "x'* and 
the back and front points plotted as "o." 
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Next, the angle of each stick vector with respect to the x-axis, Go, for each point, /, is 
determined. This may be determined by the following equation: 

00. =.H^[x2. -xl,,{y2, -yl,)-l] 
Thus, for each stick vector, the angle Go is: 



80 = 



0.898 

2.944 

-2.435 

-1.423 

-1.154 

0.902 



The vector angle of the R vector, 0i, for each index point / may be determined by the 
following equation: 

0,; = arctan[xO,. - xc,-(yOi - yc)] 
Thus, the resulting angles are: 



91 = 



-3.048 
-1.755 
-1.155 
-0.15 
0.739 
1.998 



The magnitude of the R vector, r, at point with index i may be determined by the 
following equation: 
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r.:=J^-xD.)^+(yc-yO.)^ 



The angle difference 59 (the SR Angle) for each index point i is then determined by 
subtracting 61 from 9o: 



-3.946 
-4.698 
1.28 
1.273 
1.893 
1.097 



Next, angle 02 for each index point / is determined by the equation: 



02,. =arctan[x2,. -x0i,-{y2, -yO^)] 



Next, the contour curvature, h, for each point, /, is determined. 



Where ai = 61 -02. 
Thus, the values of /i are: 



h = 



8.387 
1.061 
5.13 
-0.906 
4649 
-0.707 



After these computations, the reference table is created. For the example, the reference 
table is: 
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31.473 
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In order to aid in the description of the search process, and to avoid confusion, a subscript 
r will be added to variables h and 9o in the search procedure. 

Referring to Fig. 10, a scene image is provided, hi the same manner described above, the 
5 O contours are extracted and processed. As an example, several edge points have been selected to 
131 show computations in detail. For Fig. 10, the coordinates of the index points /, with their 
G respective back points and front points, are: 

5 
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1 
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10 These points are plotted in the graph of Fig, 11, with the index points plotted as "x" and 

the back points plotted as "o." 

Next, the angle of each stick vector with respect to the x-axis, Go, the angle 62, and the 
contour curvature hr for each point, i, is determined in the same manner described above. Thus, 
the stick vector angles 9o and the contour curvatures hr are: 
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-2.391 8.837 

-0.362 1.061 

0.565 5.13 

5 00= 1.577 hT= -0.906 

1.84 4.649 

-2.395 -0.707 



Next, the potential reference point is calculated. First, for points with a contour curvature 
10 above ±7, only one point exists at / = 0. Thus, the potential reference point and angle of rotation 

are calculated for this point: 



81, :=80„ + ffl„ 

xcfoundQ := xO^ - r^- cos ^81 ^ 

^ , J, • /n, \ xcfoundn = 381.741 
ycfound :=yO +f sin 91J o 

ycfoundg = 316.883 
rotAngla,:=80r,-B0, rotAngla, = 3.29 

For points with a contour curvature above about ±5, two points exist at / = 2 and i = 4. 
Thus, the potential reference point and angle of rotation are calculated for these points: 

91^ :=eo, + a93 



m 



91J 

■ /«, \ xcfound., = 383.297 
ycfound := yO^ + r^- sinfSl A 2 

ycfound^ = 316.689 
rotAngle,: = 80r,-80, rotAngJe, = -3.001 



9I4 :=eo, + ffl^ 

xcfound^ :=xO^ - r^' cos (^91^^ 

ycfound^ :=yO^ + r^ sinf91 J 

ycfound^ = 316.481 

rotAngle. :=90r. - 80. ^. , 

♦ 4 * totAngle^ = -2.994 



xcfound. = 382.163 

4 
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For points with a contour curvature of about ±1, three points exist at / = 1, 2 = 3, and i = 
Thus, the potential reference point and angle of rotation are calculated for these points: 



5. 
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81^ :=eo^ + aBj 



xcfound^ :=xOj - Tj COS ^91 J 
ycfoundj :=yOj + r^ sinj^Gl 



xcfoundj = 382,408 
ycfoiindj = 317.556 
fotAngle^ = 3.306 



81^ :=803 + aB^ 



xcfound^ :=x0^- r^ cos|^81^^ 
ycfound^ :=yO^ 4- r^ sin^81^J 

rotAngle^ :=80f^- 80^ 



xcfound^ = 382.199 
ycfound^ = 316.64 
rotAngle^ = 3.297 



ei^ :=803 + 59 3 



xcfound^ :'y£^- 13 003^812^ 

^ ■ ^a^ \ xcfound, = 383.257 

ycfoxmd^ :=y03 + r2 smf81J ^ 



rotAngle^ :=80r2- eO^ 



ycfound^ = 316.469 
rotAngle^ = - 3 



After these calculations, the potential reference points (xcfound^-, ycfound/) are added to 
an accumulator. Referring to Fig. 12, an image of the accumulator is provided. For the angles of 
rotation, the same process is repeated. 

The results of the computation show that the potential reference point is about (383, 317) 
and at an angle of about 3.1 radians. These results coincide with the actual reference point 
location, which is (383,318) and an angle of 318.94 = 3.298 radians. 

While the invention has been particularly shown and described by the foregoing detailed 
description, it will be understood by those skilled in the art that various other changes in form 
and detail may be made without departing from the spirit and scope of the invention. 



